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DETAILED ACTION 

Response to Arguments 
1 . Applicant's arguments filed December 4, 2006 have been fully considered but they 
are not persuasive. 

The previous Office action (mailed July 31, 2006) rejected all of the pending 
based on the combined teachings of Zawilinksi and Hoffberg. Applicant argues that the 
combination of Zawilinksi and Hoffberg is improper because "the- asserted objective is 
not found in either Zawilinski or Hoffberg." (Applicant's Reply, 10.) The Examiner 
disagrees. 

Zawilinksi discloses a system that determines a human "emotion or state of 
feeling" associated with a predetermined stimulus, e.g., a television commercial, by 
monitoring a plurality of physiological variables, including electromyography (EMG), 
heart rate, and electrodermal activity (e.g., galvanic skin response). (See col. 3, 11. 39-62, 
col. 6, 11.14-19.) Hoffberg discloses that, in addition to EMG, heart rate, and galvanic 
skin response, voice response may also be monitored to infer a user's current "mood" 
(see col. 61, 11. 14-29). Thus, Hoffberg teaches that, in addition to the variables discussed 
in Zawilinski, voice response can provide further indication as to emotional state. One of 
ordinary skill in the art would, therefore, have recognized that a microphone could be 
employed in Zawilinski 's system to detect voice input as an additional physiological 
input. Moreover, Zawilinksi discloses that laughter is a reliable indication of feeling 
pleasure. (See col. 7, 11. 35-55.) As such, one or ordinary skill in the art would also 
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recognize that the microphone detecting voice input would serve as useful indication of 
emotional state in Zawilinksi's system. 

For these reasons, the Examiner maintains that the suggestion to combine 
Zawilinski and Hoffberg is found in the references themselves and, accordingly, the 
combination is proper. Therefore, the rejections of the pending claims are maintained as 
set forth in the previous Office action. 

Claim Rejections - 35 USC § 103 

2. The text of those sections of Title 35, U.S. Code not included in this action can be 
found in a prior Office action. 

3. Claims 1-8, 10-30, and 36 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Zawilinski, U.S. Patent No. 5,676,138, in view of Hoffberg et al., U.S. 
Patent No. 6,400,996. 

Regarding claim 1, Zawilinski discloses a multimedia system comprising an 
emotional response analyzer. The system further comprises sensors for sensing the user's 
physical reactions (i.e., gaze, galvanic skin response, etc.) to various stimuli, including 
commercials, wherein said stimuli are associated with recognizable emotional responses 
(i.e., surprise, anger, disgust, etc.). Moreover, this information can be indexed and 
associated with whatever stimuli were being displayed at the time of said response. (Fig. 
1 & 2; Abstract; Col. 1, Ln. 6-47; Col. 3, Ln. 39-Col. 4, Ln. 47; Col. 5, Ln. 34-43, 63- 
Col. 6, Ln. 26, 35-Col. 7, Ln. 5; Col. 9, Ln. 50-64). But, Zawilinski fails to disclose a 
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microphone picking up vocalizations made by the user. In an analogous art, however, 
Hoffberg discloses using biometric sensor data to create a dynamic user preference 
profile, wherein voice patterns of the user are detected and associated with a recognizable 
emotional response (i.e., moods; col. 116, 11. 3-14, col. 61, 11. 14-30, col. 65, 11. 23-55; a 
microphone, or electro-acoustic transducer, is inherently present in where an acoustic 
signal (i.e., voice) is interpreted by an electrical device). Thus, it would have been 
obvious to one having ordinary skill in this art at the time of Applicant's invention to 
modify the at least one sensor of Zawilinski to include a microphone for picking up 
vocalizations, as taught by Hoffberg, to provide additional physiological data from which 
the user's preferences can be inferred. 

Claim 34 is encompassed within the rejection of Claim 1. Thus, it is analyzed and 
rejected as previously discussed. 

As to claim 2, Zawilinski and Hoffberg together disclose the claimed memory 
device (see Hoffberg, col. 1 13, 11. 47-66). 

As to claim 3, Zawilinski further discloses the system comprises a plurality of 
sensors. (Col. 6, Ln. 3-26). 

As to claim 4, Zawilinski further discloses the claimed sensor-signal receiver 
(fig. 1, item 18; col. 6, 11. 27-35). 

As to claim 5, Zawilinski further teaches the system can analyze various 
physiological data including heart rate, galvanic skin response, etc. (i.e., aggregation of 
signals; see Zawilinski as applied to claim 1, above). 
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As to claims 6-8, Zawilinski and Hoffberg together disclose the use of a camera 
(video pattern recognition; Hoffberg, col. 1 16, 11. 3-14) and video recorder recording 
images captured by the camera to an image library for comparing to video images 
received from the camera (internal database; Hoffberg, col. 59, 11. 33-61). 

As to claim 10, Hoffberg, further discloses the system comprises an environmental 
sensor for sensing a change in the viewing environment, as claimed (col. 127, 11. 5-40). 

Regarding claim 11, Zawilinski and Hoffberg together disclose the claimed 
subject matter. (See the rejection of claim 1; Hoffberg discloses determining when a 
program segment is being received that corresponds to a pre-selected viewer response 
previously associated with a physical condition status (i.e., user's mood; col. 1 16, 11. 3- 
14, 33-49, col. 118, 11. 27-43).) 

As to claim 12, Zawilinski and Hoffberg together disclose monitoring a plurality 
of viewer physical conditions (Zawilinski, col. 6, 11. 3-26). 

As to claim 13, Zawilinski further teaches the system can plot various stimuli 
changes over time (i.e., changes in condition relative to a baseline level). (Col. 9, Ln. 12- 
20). 

As to claim 14, Zawilinski and Hoffberg together disclose the claimed subject 
matter. In particular, Hoffberg discloses the physical condition is body temperature (col. 
61,11. 13-30). 

As to claim 15, Zawilinski further teaches the system can analyzed the heart rate 
of a user, (see rejection of claim 1). 
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As to claims 16 and 17, see the rejection of claims 6-8, above. 

As to Claim 18, Zawilinski and Hoffberg together disclose the claimed subject 
matter. In particular, Hoffberg discloses determining characteristics of a displayed 
programming segment, associating a viewer response corresponding to a physical 
condition with a viewer preference level, and applying the preference level to enhance 
program selection (col. 100, 11. 2-51, col. 116, 11. 33-49, col. 118, 11. 21-43). 

As to claim 19, Hoffberg further teaches providing a notification that specified 
future programming will contain at least one segment possessing the at least one 
distinguishing characteristic (col. 1 16, 1. 50 - col. 1 17, 1. 5). 

As to claim 20, Hoffberg further teaches enhancing the program selection by 
inserting a segment possessing the distinguishing characteristic (col. 1 14, 11. 12-23). 

As to claims 21 and 22, Hoffberg further discloses the characteristic is derived 
from EPG information provided with the programming, including text (col. 150, 11. 38- 
54). 

As to claim 23, Hoffberg further discloses the claimed recorder for automatically 
recording the segment corresponding to a pre-selected viewer response (see portions of 
Hoffberg cited above). 

As to claim 24, Hoffberg further discloses extracting information related to the 
program segment that corresponds to pre-selected viewer response from the 
programming, and automatically displaying the information (col. 124, 11. 45-57). 
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The limitations of claim 25 are encompassed within the portions of Zawilinski 
used to reject claim 1. 

As to claim 26, Zawilinski further teaches the system can analyzed the galvanic 
skin response of a user. (Col. 6, Ln. 10-19). 

As to claims 27 and 28, Zawilinski further teaches the system is capable of 
measuring the gaze of the viewer and the electrical activity in muscles, which relates to 
negative/positive facial expressions (i.e., visually observable response). (Col. 4, Ln. 13- 
3 1 and citations of Claim 1). As to Claim 29, Zawilinski further teaches the system 
can track the direction of the user's gaze. (Col. 9, Ln. 50-64). 

As to claim 30, Zawilinski further discloses the system is capable of analyzing the 
movement (i.e., changes in direction) of the viewers gaze and correlating the gaze 
direction with specific time periods (col. 9, 11. 50-64, col. 4, 11. 42-47). 

As to claim 36, Zawilinski and Hoffberg together disclose the method of claim 34, 
wherein an audible response is detected to infer the listener's mood. Official notice is 
taken that laughter is a well-known audible indication of mood. Thus, it would have been 
obvious to one of ordinary skill in the art at the time of the invention to modify the 
system of Zawilinski and Hoffberg to include listener laughter as an audible response, 
thereby providing an addition indication of the listener's mood. 
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4. Claims 31 and 32 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zawilinski in view of Hoffberg as applied to claim 27 above, and further in view of Black 
etal. (US Pat. No. 5,774,591). 

Claim 3 1 recites the method of claim 27; wherein the visually observable response 
includes the furrowing of the viewer's brow. As discussed above, the combined systems 
of Zawilinski and Hoffberg render obvious all limitations of Claim 27, and Zawilinski 
further teaches the system can analyzed the electrical impulses in the user's muscles (i.e., 
which can related to facial expressions), but fails to specifically recite analyzing eyebrow 
furrowing. However, within the same field of endeavor, Black et al disclose a similar 
system which analyzes changes in the viewer's eye brows. (Abstract; Col. 7, Ln. 1-45; 
Col. 26, Ln. 50-67; Col. 28, Ln. 28-62). Accordingly, it would have been obvious to one 
having ordinary skill in this art at the time of Applicant's invention to combine the 
systems of Zawilinski, Hoffberg, and Black in order to provide a system capable of 
analyzing facial features, thereby affecting the operation of a computer system. 

The limitations of claim 32 would be obvious variants of the limitations of claim 
3 1 . Since Black et al allow for the analysis of user eyebrows, analyzing the depth of 
movement in said eyebrows would only be an obvious variant. Accordingly, it would 
have been obvious to one having ordinary skill in this art at the time of Applicant's 
invention to modify the combined systems of Zawilinski, Hoffberg, and Black in order to 
provide a system which analyzes the depth of furrows, thus allowing for an analysis of 
the level of a viewer facial response (i.e., level of anger, surprise, understanding, etc). 
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5. Claims 33 and 38 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Zawilinski in view of Hoffberg as applied to claims 1 1 and 34 above, and further in view 
ofBentolilaetal. (US Pat. Pub. No. 2003/0101449). 

Zawilinski and Hoffberg together disclose the limitations of claims 1 1 and 34, but 
fail to discuss the limitations of claims 33 and 38. However, within the same field of 
endeavor, Bentolila et al disclose a similar system which utilizes a Hidden Markov 
technique. (Par. [0021]). Accordingly, it would have been obvious to one having 
ordinary skill in this art at the time of Applicant's invention to combine the systems of 
Zawilinski, Hoffberg, and Bentolila in order to provide a system for user profile data 
prediction. 

6. Claim 37 is rejected under 35 U.S.C. 103(a) as being unpatentable over Zawilinski 
in view of Hoffberg as discussed under claim 34, and further in view of Shinohara. (US 
Pat. Pub. No. 2003/0005431). 

Claim 37 recite the method of Claim 34, wherein the audibly observable response 
is the inflection (i.e., changes in pitch or tone) of a listener's vocalization, tending to 
indicate a question has been enunciated. As discussed above, the combined systems of 
Zawilinski and Hoffberg render obvious all limitations of Claim 34, but fail to 
specifically recite the limitations of Claim 37. However, within the same field of 
endeavor, Shinohara discloses a similar system which analyzes spectral data related to 
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speech patterns, such as pitch, tone, frequency, etc., in order to identify a television 
viewer. (Par. [0026]). Accordingly, it would have been obvious to one having ordinary 
skill in this art at the time of Applicant's invention to combine the systems of Zawilinski, 
Hoffberg, and Shinohara in order to provide a system which is capable of analyzing 
various tones, pitches, etc. of a given voice. 

Conclusion 

7. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the 
advisory action. In no event, however, will the statutory period for reply expire later than 
SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Chris Lambrecht whose telephone number is (571) 272- 
7297. The examiner can normally be reached on Mon-Fri, 9AM-6PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Miller can be reached on (571) 272-7353. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO 
Customer Service Representative or access to the automated information system, call 
800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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